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Abstract 

In this paper we propose an cxd— dimensional cerebellar model of neural controller 
for realistic human biodynamics. The model is developed using Feynman's action- 
amplitude (partition function) formalism. The cerebellum controller is acting as a su- 
pervisor for an autogenetic servo control of human musculo-skcletal dynamics, which 
is presented in (dissipative, driven) Hamiltonian form. The oo— dimensional cerebellar 
controller is closely related to entropic motor control. 
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Realistic human biodynamics (RHB) is a science of human (and humanoid robot) motion 
in its full complexity. It is governed by both Newtonian dynamics and biological control 
laws (see [RM ItoMI In06al IIIOeH In06c] V 

There are over 200 bones in the human skeleton driven by about 640 muscular ac- 
tuators (see, e.g., |Mar98] ). While the muscles generate driving torques in the moving 
jointsjl] subcortical neural system performs both local and global (loco)motion control: 
first reflexly controlling contractions of individual muscles, and then orchestrating all the 
muscles into synergetic actions in order to produce efficient movements. While the local 
reflex control of individual muscles is performed on the spinal control level, the global inte- 
gration of all the muscles into coordinated movements is performed within the cerebellum 
plBallnOGb] . 

All hierarchical subcortical neuro-muscular physiology, from the bottom level of a 
single muscle fiber, to the top level of cerebellar muscular synergy, acts as a temporal 
(out I in) reaction, in such a way that the higher level acts as a command/control space for 
the lower level, itself representing an abstract image of the lower one: 

1. At the muscular level, we have excitation-contraction dynamics |Hat77at IHat78t 
IHat77bj . in which (out | in) is given by the following sequence of nonlinear diffusion 
processes |II06al 11106b] : 

neural action potential synaptic potential muscular action potential 
~^ excitation contraction coupling ~^ muscle tension generating. 

^Here we need to emphasize that human joints are significantly more flexible than humanoid robot joints. 
Namely, each humanoid joint consists of a pair of coupled segments with only Eulerian rotational degrees 
of freedom. On the other hand, in each human synovial joint, besides gross Eulerian rotational movements 
(roll, pitch and yaw), we also have some hidden and restricted translations along {X,Y, Z)—a.xes. For 
example, in the knee joint, patella (knee cap) moves for about 7-10 cm from maximal extension to maximal 
flexion). It is well-known that even greater are translational amplitudes in the shoulder joint. In other 
words, within the realm of rigid body mechanics, a segment of a human arm or leg is not properly 
represented as a rigid body fixed at a certain point, but rather as a rigid body hanging on rope-like 
ligaments. More generally, the whole skeleton mechanically represents a system of fiexibly coupled rigid 
bodies. This implies the more complex kinematics, dynamics and control then in the case of humanoid 
robots. 
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Its purpose is the generation of muscular forces, to be transferred into driving torques 
within the joint anatomical geometry. 

2. At the spinal level, (out [in) is given by autogenetic-reflex stimulus-response control 
|Hou79j ■ Here we have a neural image of all individual muscles. The main purpose 
of the spinal control level is to give both positive and negative feedbacks to sta- 
bilize generated muscular forces within the 'homeostatic' (or, more appropriately, 
'homeokinetic') limits. The individual muscular actions are combined into flexor- 
extensor (or agonist-antagonist) pairs, mutually controlling each other. This is the 
mechanism of reciprocal innervation of agonists and inhibition of antagonists. It has 
a purely mechanical purpose to form the so-called equivalent muscular actuators 
(EMAs), which would generate driving torques Ti{t) for all movable joints. 

3. At the cerebellar level, (out|in) is given by sensory-motor integration |HBB96] . Here 
we have an abstracted image of all autogenetic reflexes. The main purpose of the 
cerebellar control level is integration and fine tuning of the action of all active EMAs 
into a synchronized movement, by supervising the individual autogenetic reflex cir- 
cuits. At the same time, to be able to perform in new and unknown conditions, the 
cerebellum is continuously adapting its own neural circuitry by unsupervised (self- 
organizing) learning. Its action is subconscious and automatic, both in humans and 
in animals. 

Naturally, we can ask the question: Can we assign a single (out | in) measure to all 
these neuro-muscular stimulus-response reactions? We think that we can do it; so in 
this Letter, we propose the concept of adaptive sensory-motor transition amplitude as a 
unique measure for this temporal (out|in) relation. Conceptually, this {out\m) — amplitude 
can be formulated as the ^neural path integraV: 

(out I in) = (motor I sensory) = / V[w , x] ^^^'^ . (1) 

amplitude J 

Here, the integral is taken over all activated (or, 'fired') neural pathways x^ = x*(t) of the 
cerebellum, connecting its input sensory— state with its output motor— state, symbolically 
described by adaptive neural measure "Dlw, x], defined by the weighted product (of discrete 
time steps) 

n 

T^lw, x] = lim Wi{t)dx\t), (2) 

n— >oo ^ 
t=l 

in which the synaptic weights Wi = Wi{t), included in all active neural pathways = x*(t), 
are updated by the standard learning rule 

new value{t + 1) = old value{t) + innovation{t). 
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More precisely, the weights Wi in ([2]) are updated according to one of the two standard 
neural learning schemes, in which the micro-time level is traversed in discrete steps, i.e., 
if t = to,ti, ...,tn then t + l = ti,t2, ...,tn+ill 

1. A self-organized, unsupervised (e.g., Hebbian-like |Heb49] ) learning rule: 

Wi{t + 1) = w,it) + -{wf{t) - (3) 

where a = cr^t), r] = r]{t) denote signal and noise, respectively, while superscripts d 
and a denote desired and achieved micro-states, respectively; or 

2. A certain form of a supervised gradient descent learning: 

Wi{t + 1) = Wi{t)-7]VJ{t), (4) 

where rj is a small constant, called the step size, or the learning rate, and V J{n) 
denotes the gradient of the 'performance hyper-surface' at the t— th iteration. 

Theoretically, equations ([IHll) define an oo— dimensional neural network (see |IA071lLA.Y08t 
HinSc] ). Practically, in a computer simulation we can use 10'' < n < 10^, roughly corre- 
sponding to the number of neurons in the cerebellum [IIOTal ITlOTb] . 

The exponent term S[x\ in equation ([T|) represents the autogenetic-reflex action, de- 
scribing reflexly-induced motion of all active EMAs, from their initial stimulus— state to 
their final response— state, along the family of extremal (i.e., Euler-Lagrangian) paths 
^min(*)- is properly derived in ([HHH]) below.) 

2 Sub-Cerebellar Biodynamics and Its Spinal Reflex Servo- 
Control 

Subcerebellar biodynamics includes the following three components: (i) local muscle-joint 
mechanics, (ii) whole-body musculo-skeletal dynamics, and (iii) autogenetic reflex servo- 
control. 

2.1 Local Muscle— Joint Mechanics 

Local muscle-joint mechanics comprises of [Iva06t ITl06a^ 11106b] ): 

^Note that we could also use a reward-based, reinforcement learning rule |SB98] . in which system learns 
its optimal policy: 

innovation{t) — \re'ward{t) — penalty {t)\. 
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1. Synovial joint dynamics, giving the first stabilizing effect to the conservative skeleton 
dynamics, is described by the (x, x)-form of the Rayleigh - Van der Pol's dissipation 
function 

n 
i=l 

where Oj and /3j denote dissipation parameters. Its partial derivatives give rise to the 
viscous-damping torques and forces in the joints 

which are linear in and quadratic in x*. 

2. Muscular dynamics, giving the driving torques and forces ^?""*'=ie _ jrmuscle^j.^ 
with (i = 1, . . . ,n) for RHB, describes the internal excitation and contraction dynamics 
of equivalent muscular actuators [Hat 78] . 

(a) Excitation dynamics can be described by an impulse force-time relation 

pimp ^ F^{1 - e-*/^') if stimulation >0 
pimp ^ F^e-^/n if stimulation =0, 

where F-* denote the maximal isometric muscular torques and forces, while Tj denote the 
associated time characteristics of particular muscular actuators. This relation represents 
a solution of the Wilkie's muscular active-state element equation |Wil56] 

fi + -ffj, = jSA, /i(0) = 0, 0<5<1, 

where fi = n{t) represents the active state of the muscle, 7 denotes the element gain, 
A corresponds to the maximum tension the element can develop, and S = S{r) is the 
'desired' active state as a function of the motor unit stimulus rate r. This is the basis for 
the RHB force controller. 

(b) Contraction dynamics has classically been described by the Hill's hyperbolic force- 
velocity relation |Hil38| 

pHiii _ {Ffhj — 6ijaiX^ ) 
' ~ {SijX^ + hi) ' 

where a, and 6j denote the Hill's parameters, corresponding to the energy dissipated during 
the contraction and the phosphagenic energy conversion rate, respectively, while 5ij is the 
Kronecker's 5— tensor. 

In this way, RHB describes the excitation/contraction dynamics for the ith equivalent 
muscle-joint actuator, using the simple impulse-hyperbolic product relation 
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Now, for the purpose of biomedical engineering and rehabilitation, RHB has developed 
the so-called hybrid rotational actuator. It includes, along with muscular and viscous 
forces, the D.C. motor drives, as used in robotics |VBS90t llvaOGl In06a| 

H"'" = ik{t) - JkXkit) - BkXkit), 

with 

lkik{t) + Rkik{t) + CkXkit) = Uk{t), 

where k = l,...,n, ik{t) and Uk{t) denote currents and voltages in the rotors of the 
drives, Rk,lk and Ck are resistances, inductances and capacitances in the rotors, respec- 
tively, while Jk and Bk correspond to inertia moments and viscous dampings of the drives, 
respectively. 

Finally, to make the model more realistic, we need to add some stochastic torques and 
forces (iSUn HIOTa] 

J^^^'"''' = Bij[x\t),t] dW^it) 

where Bij[x{t),t] represents continuous stochastic diffusion fluctuations, and W^{t) is an 
iV— variable Wiener process (i.e. generalized Brownian motion), with dW^{t) = W^{t + 
dt)-W^{t) for j = l,...,N. 



2.2 Hamiltonian Biodynamics and Its Reflex Servo— Control 



General form of Hamiltonian biodynamics on the configuration manifold of human motion 
is formulated in [ISOTl |Iva02l H^aM llBOSl HIOGal In06c] 1 using the concept of Euclidean 
group of motions SE(3jj (see Figured]), 

Briefly, based on affine Hamiltonian function of human motion, formally Ha : T*Q ^ 
M, in local canonical coordinates on the symplectic phase space (which is the cotangent 



■^Briefly, the Euclidean SE(3)-group is defined as a semidirect (noncommutative) product of 3D rotations 
and 3D translations, SE{3) := SO{3) > R^. Its most important subgroups are the following (for technical 



details see [1106?, 'PC05', HlOTc]): 



Subgroup 



Definition 



5*0(3), group of rotations 
in 3D (a spherical joint) 



Set of all proper orthogonal 
3 X 3 — rotational matrices 



SE{2), special Euclidean group 
in 2D (all planar motions) 



Set of all 3 X 3 — matrices: 
cos sin rj_ 



SO{2), group of rotations in 2D 
subgroup of S'i?(2)-group 
(a revolute joint) 



— sm f 




cos f 




Set of all proper orthogonal 
2 X 2 — rotational matrices 
included in SE{2) — group 



, group of translations in 3D 
(all spatial displacements) 



Euclidean 3D vector space 
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Figure 1: The configuration manifold Q of the human musculoskeletal dynamics is defined 
as an anthropomorphic product of constrained Euclidean SE(3)-groups acting in all major 
(synovial) human joints. 

bundle of the human configuration manifold Q) T*Q given as 

Ha{x,p,u) = Ho{x,p) - W{x,p)uj, (5) 

where Hq{x,p) = Ek{p) + Ep{x) is the physical Hamiltonian (kinetic + potential energy) 
dependent on joint coordinates and their canonical momenta pi, = H^{x,p), {j = 
1, . . . , m < n are the coupling Hamiltonians corresponding to the system's active joints and 
Ui = Ui{t, x,p) are (reflex) feedback-controls. Using ([5]) we come to the affine Hamiltonian 
control RHB-system, in deterministic form 

±' = dp^ Ho - dp^ W uj +dp^R, (6) 
Pi = Ti — dr^iHo + d^iH^ Uj + d^iR, 

o' = -du,Ha = W, 

x\0)=xl,, K(0)=pO, 

{i = l,...,n; j = 1,..., Q <n), 

(where du = d/du, Ti = J^i{t,x,p), Hq = Ho{x,p), W = W{x,p), Ha = Ha{x,p,u), 
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R = R{x,p)), as well as in the fuzzy-stochastic form jlSOU HIOTa] 



dq' = {dp^Ho{a^) - dpW{ai,) uj + dp^R) dt, 

dpi = Bij[x\t),t]dW^it) + (7) 

{Ti - d^rHQ{a^) + ^^^W{ap) uj + d^rR) dt, 
do' = -du^Haia^) dt = W{a^) dt, 

In (HI)-©, R = R{x,p) denotes the joint (nonlinear) dissipation function, are affine 
system outputs (which can be different from joint coordinates); {fx}^ (with /u > 1) denote 
fuzzy sets of conservative parameters (segment lengths, masses and moments of inertia), 
dissipative joint dampings and actuator parameters (amplitudes and frequencies), while 
the bar (.) over a variable denotes the corresponding fuzzified variable; Bij\q^{t),t] denote 
diffusion fluctuations and W^{t) are discontinuous jumps as the n-dimensional Wiener 
process. 

In this way, the force RHB servo-controller is formulated as affine control Hamiltonian- 
systems ([GHZl)) which resemble the autogenetic motor servo (see Appendix), acting on the 
spinal-reflex level of the human locomotion control. A voluntary contraction force F of 
human skeletal muscle is reflexly excited (positive feedback +F~^) by the responses of 
its spindle receptors to stretch and is reflexly inhibited (negative feedback —F~^) by the 
responses of its Golgi tendon organs to contraction. Stretch and unloading reflexes are 
mediated by combined actions of several autogenetic neural pathways, forming the so- 
called 'motor servo.' The term 'autogenetic' means that the stimulus excites receptors 
located in the same muscle that is the target of the reflex response. The most important 
of these muscle receptors are the primary and secondary endings in the muscle-spindles, 
which are sensitive to length change - positive length feedback +F~^, and the Golgi 
tendon organs, which are sensitive to contractile force - negative force feedback —F~^. 

The gain G of the length feedback +F~^ can be expressed as the positional stiffness 
(the ratio G ~ 5 = dF/dx of the force-F change to the length-x change) of the muscle 
system. The greater the stiffness S, the less the muscle will be disturbed by a change 
in load. The autogenetic circuits -\-F~^ and —F~^ appear to function as servoregulatory 
loops that convey continuously graded amounts of excitation and inhibition to the large 
(alpha) skeletomotor neurons. Small (gamma) fusimotor neurons innervate the contractile 
poles of muscle spindles and function to modulate spindle-receptor discharge. 
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3 Cerebellum: The Adaptive Path— Integral Comparator 



3.1 Cerebellum as a Neural Controller 

Having, thus, defined the spinal reflex control level, we proceed to model the top subcorti- 
cal commander /controller, the cerebellum (see Appendix). The cerebellum is responsible 
for coordinating precisely timed (out | in) activity by integrating motor output with ongo- 
ing sensory feedback (see Figure [2]). It receives extensive projections from sensory-motor 
areas of the cortex and the periphery and directs it back to premotor and motor cortex 
[GheQOl IGheOlj . This suggests a role in sensory-motor integration and the timing and 
execution of human movements. The cerebellum stores patterns of motor control for fre- 
quently performed movements, and therefore, its circuits are changed by experience and 
training. It was termed the adjustable pattern generator in the work of J. Houk and col- 
laborators |HBB96j . Also, it has become the inspiring 'brain-model' in robotic research 
[S^[Sdl98l[Sdl99] . 
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Figure 2: Schematic (out|in) organization of the primary cerebellar circuit. In essence, 
excitatory inputs, conveyed by collateral axons of Mossy and Climbing fibers activate di- 
rectly neurones in the Deep cerebellar nuclei. The activity of these latter is also modulated 
by the inhibitory action of the cerebellar cortex, mediated by the Purkinje cells. 

The cerebellum is known to be involved in the production and learning of smooth 
coordinated movements jTGK92t IFSB97j . Two classes of inputs carry information into the 
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cerebellum: the mossy fibers (MFs) and the climbing fibers (CFs). The MFs provide both 
plant state and contextual information |BC81j . The CFs, on the other hand, are thought 
to provide information that reflect errors in recently generated movements [ Ito841 IIto90| . 
This information is used to adjust the programs encoded by the cerebellum. The MFs 
carry plant state, motor efference, and other contextual signals into the cerebellum. These 
fibers impinge on granule cells, whose axons give rise to parallel fibers (PFs). Through 
the combination of inputs from multiple classes of MFs and local inhibitory interneurons, 
the granule cells are thought to provide a sparse expansive encoding of the incoming state 
information |Alb71] . The large number of PFs converge on a much smaller set of Purkinje 
cells (PCs), while the PCs, in turn, provide inhibitory signals to a single cerebellar nuclear 
cell [FSB97j . Using this principle, the Cerebellar Model Arithmetic Computer, or CMAC- 
neural network has been built |Alb71t[MGK92j and implemented in robotics |Sma98j . using 
trial-and-error learning to produce bursts of muscular activity for controlling robot arms. 

So, this 'cerebellar control' works for simple robotic problems, like non-redundant 
manipulation. However, comparing the number of its neurons (10^ - 10^), to the size 
of conventional neural networks (including CMAC), suggests that artificial neural nets 
cannot satisfactorily model the function of this sophisticated 'super-bio~computer', as its 
dimensionality is virtually infinite. Despite a lot of research dedicated to its structure and 
function (see [HBB96] and references there cited), the real nature of the cerebellum still 
remains a 'mystery'. 




Figure 3: The cerebellum as a motor controller. 

The main function of the cerebellum as a motor controller is depicted in Figure [3l A 
coordinated movement is easy to recognize, but we know little about how it is achieved. 
In search of the neural basis of coordination, a model of spinocerebellar interactions was 
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recently presented in |AG05j . in which the structural and functional organizing principle 
is a division of the cerebellum into discrete micro-complexes. Each micro-complex is the 
recipient of a specific motor error signal, that is, a signal that conveys information about an 
inappropriate movement. These signals are encoded by spinal reflex circuits and conveyed 
to the cerebellar cortex through climbing fibre afferents. This organization reveals salient 
features of cerebellar information processing, but also highlights the importance of systems 
level analysis for a fuller understanding of the neural mechanisms that underlie behavior. 



3.2 Hamiltonian Action and Neural Path Integral 

Here, we propose a quantum-like adaptive control approach to modeling the 'cerebellar 
mystery'. Corresponding to the affine Hamiltonian control function ([5]) we define the affine 
Hamiltonian control action, 



rtout 

Saff[q,p] = dr [pici - Haff{q,p)] . (8) 

J tin 



From the affine Hamiltonian action ^ we further derive the associated expression for 
the neural phase-space path integral (in normal units), representing the cerebellar sensory- 
motor amplitude (out | in), 

(?Lt,Pr*ICPr) = [v[w,q,p]e''^ff^'^'P^ (9) 



f rtout 

V[w,q,p] ex.pl i / dr [piq' - Haff{q,p)] 

^ tin 



with J V[w,q,p] = jfl 



w'{T)dpi{T)dq'{T) 



2n 

T = l 

where Wi = Wi{t) denote the cerebellar synaptic weights positioned along its neural path- 
ways, being continuously updated using the Hebbian-like self-organizing learning rule 
(l3|). Given the transition amplitude out\in ([9]), the cerebellar sensory-motor transition 
probability is defined as its absolute square, |(out|in)p. 

In the phase-space path integral (jS]), qj^ = q\^{t), ql^t = qlutit); vT = P^^ = 

pout^^y^ t^^ < t < tout, for all discrete time steps, t = 1, ...,n — > oo, and we are allowing 
for the affine Hamiltonian Haff{q,p) to depend upon all the (M < N) EMA-angles 
and angular momenta collectively. Here, we actually systematically took a discretized 
differential time limit of the form t^ — t^-i = dr (both a and r denote discrete time steps) 

and wrote = For technical details regarding the path integral calculations on 

Riemannian and symplectic manifolds (including the standard regularization procedures), 
see ^KlaOTllKlaOO] . 
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Now, motor learning occurring in the cerebellum can be observed using functional MR 
imaging, showing changes in the cerebellar action potential, related to the motor tasks 
(see, e.g., |MA02) ). To account for these electro-physiological currents, we need to add 
the source term Ji{t)q^[t) to the afiine Hamiltonian action ([8]), (the current Jj = Ji{t) acts 
as a source JiA^ of the cerebellar electrical potential = A^{t)), 



Saff[q,P,J] = 




dr [pi(f - Haff{q,p) + Jiq'] , 



which, subsequently gives the cerebellar path integral with the action potential source, 
coming either from the motor cortex or from other subcortical areas. 

Note that the standard Wick rotation: t t (see |Kla97t IKlaOO| ). makes our path 
integral real, i.e., 

J V[w,q,p]e'^''ff^'''P^ Wick j V[w,q,p]e- ^-ff^'i'P\ 

while their subsequent discretization gives the standard thermodynamic partition function 
(see Appendix), 

Z = Y^--'^^'I^, (10) 

3 

where is the energy eigenvalue corresponding to the affine Hamiltonian Haff{q,p), T is 
the temperature-like environmental control parameter, and the sum runs over all energy 
eigenstates (labelled by the index j). From (|10p . we can further calculate all statistical 
and thermodynamic system properties (see [Fey72] ) , as for example, transition entropy 
S = ks^T^ Z, etc. 

3.3 Entropy and Motor Control 

Our cerebellar path integral controller is closely related to entropic motor control jHNOSat 
IHNOSbj . which deals with neuro-physiological feedback information and environmental 
uncertainty. The probabilistic nature of human motor action can be characterized by en- 
tropies at the level of the organism, task, and environment. Systematic changes in motor 
adaptation are characterized as task-organism and environment -organism tradeoffs in en- 
tropy. Such compensatory adaptations lead to a view of goal-directed motor control as the 
product of an underlying conservation of entropy across the task-organism-environment 
system. In particular, an experiment conducted in |HN08bj examined the changes in en- 
tropy of the coordination of isometric force output under different levels of task demands 
and feedback from the environment. The goal of the study was to examine the hypothesis 
that human motor adaptation can be characterized as a process of entropy conservation 
that is reflected in the compensation of entropy between the task, organism motor output. 
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and environment. Information entropy of the coordination dynamics relative phase of the 
motor output was made conditional on the ideahzed situation of human movement, for 
which the goal was always achieved. Conditional entropy of the motor output decreased 
as the error tolerance and feedback frequency were decreased. Thus, as the hkelihood of 
meeting the task demands was decreased increased task entropy and/or the amount of in- 
formation from the environment is reduced increased environmental entropy, the subjects 
of this experiment employed fewer coordination patterns in the force output to achieve the 
goal. The conservation of entropy supports the view that context dependent adaptations 
in human goal-directed action are guided fundamentally by natural law and provides a 
novel means of examining human motor behavior. This is fundamentally related to the 
Heisenherg uncertainty principle [II08b] and further supports the argument for the primacy 
of a probabilistic approach toward the study of biodynamic cognition systems. 

The action-amplitude formalism represents a kind of a generalization of the Haken- 
Kelso-Bunz (HKB) model of self-organization in the individual's motor system |HKB85( 
IKel95j . including: multi-stability, phase transitions and hysteresis effects, presenting a 
contrary view to the purely feedback driven systems. HKB uses the concepts of synergetics 
(order parameters, control parameters, instability, etc) and the mathematical tools of 
nonlinearly coupled (nonlinear) dynamical systems to account for self-organized behavior 
both at the cooperative, coordinative level and at the level of the individual coordinating 
elements. The HKB model stands as a building block upon which numerous extensions 
and elaborations have been constructed. In particular, it has been possible to derive it 
from a realistic model of the cortical sheet in which neural areas undergo a reorganization 
that is mediated by intra- and inter-cortical connections. Also, the HKB model describes 
phase transitions ('switches') in coordinated human movement as follows: (i) when the 
agent begins in the anti-phase mode and speed of movement is increased, a spontaneous 
switch to symmetrical, in-phase movement occurs; (ii) this transition happens swiftly at 
a certain critical frequency; (iii) after the switch has occurred and the movement rate is 
now decreased the subject remains in the symmetrical mode, i.e. she does not switch 
back; and (iv) no such transitions occur if the subject begins with symmetrical, in-phase 
movements. The HKB dynamics of the order parameter relative phase as is given by a 
nonlinear first-order ODE: 

(j) = {a + 2/?r^) sin cj) — (3r'^ sin 2(1), 

where (j) is the phase relation (that characterizes the observed patterns of behavior, changes 
abruptly at the transition and is only weakly dependent on parameters outside the phase 
transition), r is the oscillator amplitude, while a, (3 are coupling parameters (from which 
the critical frequency where the phase transition occurs can be calculated). 
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4 Appendix 



4.1 Houk's Autogenetic Motor Servo 

About three decades ago, James Houk pointed out in |Hou67t fHSGTOl lHou78t IHou79| that 
stretch and unloading reflexes were mediated by combined actions of several autogenetic 
neural pathways. In this context, "autogenetic" (or, autogenic) means that the stimulus 
excites receptors located in the same muscle that is the target of the reflex response. 
The most important of these muscle receptors are the primary and secondary endings 
in muscle spindles, sensitive to length change, and the Golgi tendon organs, sensitive to 
contractile force. The autogenetic circuits appear to function as servo-regulatory loops 
that convey continuously graded amounts of excitation and inhibition to the large (alpha) 
skeletomotor neurons. Small (gamma) fusimotor neurons innervate the contractile poles of 
muscle spindles and function to modulate spindle-receptor discharge. Houk's term "motor 
servo" |Hou78j has been used to refer to this entire control system, summarized by the 
block diagram in Figure HI 
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Figure 4: Houk's autogenetic motor servo. 
Prior to a study by Matthews |Mat69] , it was widely assumed that secondary endings 
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belong to the mixed population of "flexor reflex afferents," so called because their activa- 
tion provokes the flexor reflex pattern - excitation of flexor motoneurons and inhibition of 
extensor motoneurons. Matthews' results indicated that some category of muscle stretch 
receptor other than the primary ending provides important excitation to extensor muscles, 
and he argued forcefully that it must be the secondary ending. 

The primary and secondary muscle spindle afferent fibers both arise from a specialized 
structure within the muscle, the muscle spindle, a fusiform structure 4-7 mm long and 80- 
200 /X in diameter. The spindles are located deep within the muscle mass, scattered widely 
through the muscle body, and attached to the tendon, the endomysium or the perimysium, 
so as to be in parallel with the extrafusal or regular muscle fibers. Although spindles are 
scattered widely in muscles, they are not found throughout. Muscle spindle (see Figure 
??) contains two types of intrafusal muscle fibers (intrafusal means inside the fusiform 
spindle): the nuclear bag fibers and the nuclear chain fibers. The nuclear bag fibers are 
thicker and longer than the nuclear chain fibers, and they receive their name from the 
accumulation of their nuclei in the expanded bag-like equatorial region-the nuclear bag. 
The nuclear chain fibers have no equatorial bulge; rather their nuclei are lined up in the 
equatorial region-the nuclear chain. A typical spindle contains two nuclear bag fibers and 
4-5 nuclear chain fibers. 

The pathways from primary and secondary endings are treated commonly by Houk 
in Figure [U since both receptors are sensitive to muscle length and both provoke reflex 
excitation. However, primary endings show an additional sensitivity to the dynamic phase 
of length change, called dynamic responsiveness, and they also show a much-enhanced 
sensitivity to small changes in muscle length [Mat 72] . 

The motor servo comprises three closed circuits (Figure |4]), two neural feedback path- 
ways, and one circuit representing the mechanical interaction between a muscle and its 
load. One of the feedback pathways, that from spindle receptors, conveys information 
concerning muscle length, and it follows that this loop will act to keep muscle length 
constant. The other feedback pathway, that from tendon organs, conveys information 
concerning muscle force, and it acts to keep force constant. 

In general, it is physically impossible to maintain both muscle length and force con- 
stant when external loads vary; in this situation the action of the two feedback loops will 
oppose each other. For example, an increased load force will lengthen the muscle and cause 
muscular force to increase as the muscle is stretched out on its length-tension curve. The 
increased length will lead to excitation of motoneurons, whereas the increased force will 
lead to inhibition. It follows that the net regulatory action conveyed by skeletomotor out- 
put will depend on some relationship between force change and length change and on the 
strength of the feedback from muscle spindles and tendon organs. A simple mathematical 
derivation |NH76j demonstrates that the change in skeletomotor output, the error signal 
of the motor servo. Should be proportional to the difference between a regulated stiffness 
and the actual stiffness provided by the mechanical properties of the muscle, where stiff- 
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ness has the units of force change divided by length change. The regulated stiffness is 
determined by the ratio of the gain of length to force feedback. 

It follows that the combination of spindle receptor and tendon organ feedback will 
tend to maintain the stiffness of the neuromuscular apparatus at some regulated level. If 
this level is high, due to a high gain of length feedback and a low gain of force feedback, 
one could simply forget about force feedback and treat muscle length as the regulated 
variable of the system. However, if the regulated level of stiffness is intermediate in value, 
i.e. not appreciably different from the average stiffness arising from muscle mechanical 
properties in the absence of reflex actions, one would conclude that stiffness, or its inverse, 
compliance, is the regulated property of the motor servo. 

In this way, the autogenetic reflex motor servo provides the local, reflex feedback 
loops for individual muscular contractions. A voluntary contraction force F of human 
skeletal muscle is reflexly excited (positive feedback +F~^) by the responses of its spindle 
receptors to stretch and is reflexly inhibited (negative feedback —F~^) by the responses 
of its Golgi tendon organs to contraction. Stretch and unloading reflexes are mediated by 
combined actions of several autogenetic neural pathways, forming the motor servo (see 
[II06al[II06blln06c] ^. 

In other words, branches of the afferent fibers also synapse with with interneurons 
that inhibit motor neurons controlling the antagonistic muscles - reciprocal inhibition. 
Consequently, the stretch stimulus causes the antagonists to relax so that they cannot 
resists the shortening of the stretched muscle caused by the main reflex arc. Similarly, 
firing of the Golgi tendon receptors causes inhibition of the muscle contracting too strong 
and simultaneous reciprocal activation of its antagonist. 

4.2 Cerebellum and Muscular Synergy 

The cerebellum is a brain region anatomically located at the bottom rear of the head 
(the hindbrain), directly above the brainstem, which is important for a number of subcon- 
scious and automatic motor functions, including motor learning. It processes information 
received from the motor cortex, as well as from proprioceptors and visual and equilibrium 
pathways, and gives 'instructions' to the motor cortex and other subcortical motor centers 
(like the basal nuclei), which result in proper balance and posture, as well as smooth, co- 
ordinated skeletal movements, like walking, running, jumping, driving, typing, playing the 
piano, etc. Patients with cerebellar dysfunction have problems with precise movements, 
such as walking and balance, and hand and arm movements. The cerebellum looks simi- 
lar in all animals, from fish to mice to humans. This has been taken as evidence that it 
performs a common function, such as regulating motor learning and the timing of move- 
ments, in all animals. Studies of simple forms of motor learning in the vestibulo-ocular 
reflex and eye-blink conditioning are demonstrating that timing and amplitude of learned 
movements are encoded by the cerebellum. 
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When someone compares learning a new skill to learning how to ride a bike they imply 
that once mastered, the task seems imbedded in our brain forever. Well, imbedded in the 
cerebellum to be exact. This brain structure is the commander of coordinated movement 
and possibly even some forms of cognitive learning. Damage to this area leads to motor 
or movement difficulties. 

A part of a human brain that is devoted to the sensory-motor control of human move- 
ment, that is motor coordination and learning, as well as equilibrium and posture, is the 
cerebellum (which in Latin means "little brain"). It performs integration of sensory per- 
ception and motor output. Many neural pathways link the cerebellum with the motor 
cortex, which sends information to the muscles causing them to move, and the spino- 
cerebellar tract, which provides proprioception, or feedback on the position of the body 
in space. The cerebellum integrates these pathways, using the constant feedback on body 
position to fine-tune motor movements |Ito84] . 

The human cerebellum has 7-14 million Purkinje cells. Each receives about 200,000 
synapses, most onto dendritic splines. Granule cell axons form the parallel fibers. They 
make excitatory synapses onto Purkinje cell dendrites. Each parallel fibre synapses on 
about 200 Purkinje cells. They create a strip of excitation along the cerebellar folia. 

Mossy fibers are one of two main sources of input to the cerebellar cortex (see Figure 
[5]). A mossy fibre is an axon terminal that ends in a large, bulbous swelling. These mossy 
fibers enter the granule cell layer and synapse on the dendrites of granule cells; in fact the 
granule cells reach out with little 'claws' to grasp the terminals. The granule cells then 
send their axons up to the molecular layer, where they end in a T and run parallel to the 
surface. For this reason these axons are called parallel fibers. The parallel fibers synapse 
on the huge dendritic arrays of the Purkinje cells. However, the individual parallel fibers 
are not a strong drive to the Purkinje cells. The Purkinje cell dendrites fan out within a 
plane, like the splayed fingers of one hand. If we were to turn a Purkinje cell to the side, it 
would have almost no width at all. The parallel fibers run perpendicular to the Purkinje 
cells, so that they only make contact once as they pass through the dendrites. 

Unless firing in bursts, parallel fibre EPSPs do not fire Purkinje cells. Parallel fibers 
provide excitation to all of the Purkinje cells they encounter. Thus, granule cell activity 
results in a strip of activated Purkinje cells. 

Mossy fibers arise from the spinal cord and brainstem. They synapse onto granule cells 
and deep cerebellar nuclei. The Purkinje cell makes an inhibitory synapse (GABA) to the 
deep nuclei. Mossy fibre input goes to both cerebellar cortex and deep nuclei. When the 
Purkinje cell fires, it inhibits output from the deep nuclei. 

The climbing fibre arises from the inferior olive. It makes about 300 excitatory synapses 
onto one Purkinje cell. This powerful input can fire the Purkinje cell. 

The parallel fibre synapses are plastic — that is, they can be modified by experience. 
When parallel fibre activity and climbing fibre activity converge on the same Purkinje cell, 
the parallel fibre synapses become weaker (EPSPs are smaller). This is called long-term 
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<tUinbing fiber 

Figure 5: Stereotypical ways throughout the cerebellum. 

depression. Weakened parallel fibre synapses result in less Purkinje cell activity and less 
inhibition to the deep nuclei, resulting in facilitated deep nuclei output. Consequently, 
the mossy fibre collaterals control the deep nuclei. 

The basket cell is activated by parallel fibers afferents. It makes inhibitory synapses 
onto Purkinje cells. It provides lateral inhibition to Purkinje cells. Basket cells inhibit 
Purkinje cells lateral to the active beam. 

Golgi cells receive input from parallel fibers, mossy fibers, and climbing fibers. They 
inhibit granule cells. Golgi cells provide feedback inhibition to granule cells as well as 
feedforward inhibition to granule cells. Golgi cells create a brief burst of granule cell 
activity. 

Although each parallel fibre touches each Purkinje cell only once, the thousands of 
parallel fibers working together can drive the Purkinje cells to fire like mad. 

The second main type of input to the folium is the climbing fibre. The climbing fibers 
go straight to the Purkinje cell layer and snake up the Purkinje dendrites, like ivy climbing 
a trellis. Each climbing fibre associates with only one Purkinje cell, but when the climbing 
fibre fires, it provokes a large response in the Purkinje cell. 

The Purkinje cell compares and processes the varying inputs it gets, and finally sends 
its own axons out through the white matter and down to the deep nuclei. Although the 
inhibitory Purkinje cells are the main output of the cerebellar cortex, the output from the 
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cerebellum as a whole comes from the deep nuclei. The three deep nuclei are responsible 
for sending excitatory output back to the thalamus, as well as to postural and vestibular 
centers. 

There are a few other cell types in cerebellar cortex, which can all be lumped into the 
category of inhibitory interneuron. The Golgi cell is found among the granule cells. The 
stellate and basket cells live in the molecular layer. The basket cell (right) drops axon 
branches down into the Purkinje cell layer where the branches wrap around the cell bodies 
like baskets. 

The cerebellum operates in 3's: there are 3 highways leading in and out of the cere- 
bellum, there are 3 main inputs, and there are 3 main outputs from 3 deep nuclei. They 
are: 

The 3 highways are the peduncles. There are 3 pairs (see |Mol971 IHar971 IMar98| ) : 

1. The inferior cerebellar peduncle (restiform body) contains the dorsal spinocerebellar 
tract (DSCT) fibers. These fibers arise from cells in the ipsilateral Clarke's col- 
umn in the spinal cord (C8~L3). This peduncle contains the cuneo-cerebellar tract 
(CCT) fibers. These fibers arise from the ipsilateral accessory cuneate nucleus. The 
largest component of the inferior cerebellar peduncle consists of the olivo-cerebellar 
tract (OCT) fibers. These fibers arise from the contralateral inferior olive. Finally, 
vestibulo-cerebellar tract (VCT) fibers arise from cells in both the vestibular gan- 
glion and the vestibular nuclei and pass in the inferior cerebellar peduncle to reach 
the cerebellum. 

2. The middle cerebellar peduncle (brachium pontis) contains the pontocerebellar tract 
(PCT) fibers. These fibers arise from the contralateral pontine grey. 

3. The superior cerebellar peduncle (brachium conjunctivum) is the primary efferent 
(out of the cerebellum) peduncle of the cerebellum. It contains fibers that arise 
from several deep cerebellar nuclei. These fibers pass ipsilaterally for a while and 
then cross at the level of the inferior colliculus to form the decussation of the superior 
cerebellar peduncle. These fibers then continue ipsilaterally to terminate in the red 
nucleus ('ruber-duber') and the motor nuclei of the thalamus (VA, VL). 

The 3 inputs are: mossy fibers from the spinocerebellar pathways, climbing fibers from 
the inferior olive, and more mossy fibers from the pons, which are carrying information 
from cerebral cortex (see Figure [6]). The mossy fibers from the spinal cord have come up 
ipsilaterally, so they do not need to cross. The fibers coming down from cerebral cortex, 
however, do need to cross (as the cerebrum is concerned with the opposite side of the 
body, unlike the cerebellum). These fibers synapse in the pons (hence the huge block of 
fibers in the cerebral peduncles labelled 'cortico-pontine'), cross, and enter the cerebellum 
as mossy fibers. 
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Figure 6: Inputs and outputs of the cerebellum. 

The 3 deep nuclei are the fastigial, interposed, and dentate nuclei. The fastigial nu- 
cleus is primarily concerned with balance, and sends information mainly to vestibular and 
reticular nuclei. The dentate and interposed nuclei are concerned more with voluntary 
movement, and send axons mainly to thalamus and the red nucleus. 

The main function of the cerebellum as a motor controller is depicted in Figure [3l A 
coordinated movement is easy to recognize, but we know little about how it is achieved. 
In search of the neural basis of coordination, a model of spinocerebellar interactions was 
recently presented in |AG05j . in which the structure- functional organizing principle is 
a division of the cerebellum into discrete micro-complexes. Each micro-complex is the 
recipient of a specific motor error signal, that is, a signal that conveys information about an 
inappropriate movement. These signals are encoded by spinal reflex circuits and conveyed 
to the cerebellar cortex through climbing fibre afferents. This organization reveals salient 
features of cerebellar information processing, but also highlights the importance of systems 
level analysis for a fuller understanding of the neural mechanisms that underlie behavior. 

The authors of [A G05j reviewed anatomical and physiological foundations of cerebellar 
information processing. The cerebellum is crucial for the coordination of movement. The 
authors presented a model of the cerebellar paravermis, a region concerned with the control 
of voluntary limb movements through its interconnections with the spinal cord. They 
particularly focused on the olivo-cerebellar climbing fibre system. 
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Climbing fibres are proposed to convey motor error signals (signals that convey in- 
formation about inappropriate movements) related to elementary limb movements that 
result from the contraction of single muscles. The actual encoding of motor error signals 
is suggested to depend on sensorimotor transformations carried out by spinal modules that 
mediate nociceptive withdrawal reflexes. 

The termination of the climbing fibre system in the cerebellar cortex subdivides the 
paravermis into distinct microzones. Functionally similar but spatially separate micro- 
zones converge onto a common group of cerebellar nuclear neurons. The processing units 
formed as a consequence are termed 'multizonal micro-complexes' (MZMCs), and are each 
related to a specific spinal reflex module. 

The distributed nature of microzones that belong to a given MZMC is proposed to 
enable similar climbing fibre inputs to integrate with mossy fibre inputs that arise from 
different sources. Anatomical results consistent with this notion have been obtained. 

Within an individual MZMC, the skin receptive fields of climbing fibres, mossy fibres 
and cerebellar cortical inhibitory interneurons appear to be similar. This indicates that 
the inhibitory receptive fields of Purkinje cells within a particular MZMC result from the 
activation of inhibitory interneurons by local granule cells. 

On the other hand, the parallel fibre-mediated excitatory receptive fields of the Purk- 
inje cells in the same MZMC differ from all of the other receptive fields, but are similar 
to those of mossy fibres in another MZMC. This indicates that the excitatory input to 
Purkinje cells in a given MZMC originates in non-local granule cells and is mediated over 
some distance by parallel fibres. 

The output from individual MZMCs often involves two or three segments of the ipsi- 
lateral limb, indicative of control of multi-joint muscle synergies. The distal-most muscle 
in this synergy seems to have a roughly antagonistic action to the muscle associated with 
the climbing fibre input to the MZMC. 

The model proposed in |AG05j indicates that the cerebellar paravermis system could 
provide the control of both single- and multi-joint movements. Agonist-antagonist activity 
associated with single-joint movements might be controlled within a particular MZMC, 
whereas coordination across multiple joints might be governed by interactions between 
MZMCs, mediated by parallel fibres. 

Two main theories address the function of the cerebellum, both dealing with motor 
coordination. One claims that the cerebellum functions as a regulator of the "timing 
of movements." This has emerged from studies of patients whose timed movements are 
disrupted |IKD88j . 

The second, "Tensor Network Theory" provides a mathematical model of transforma- 
tion of sensory (covariant) space-time coordinates into motor (contravariant) coordinates 
by cerebellar neuronal networks [PLSOj lPL82l IPL85] . 

Studies of motor learning in the vestibulo-ocular reflex and eye-blink conditioning 
demonstrate that the timing and amplitude of learned movements are encoded by the 
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cerebellum [BK04| . Many synaptic plasticity mechanisms have been found throughout the 
cerebellum. The Marr-Albus model mostly attributes motor learning to a single plasticity 
mechanism: the long-term depression of parallel fiber synapses. The Tensor Network 
Theory of sensory-motor transformations by the cerebellum has also been experimentally 
supported |GZ86j . 

4.3 Feynman's Partition Function 

Recall that in statistical mechanics, the so-called partition function Z is a quantity that 
encodes the statistical properties of a system in thermodynamic equilibrium. It is a func- 
tion of temperature and other parameters, such as the volume enclosing a gas. Other 
thermodynamic variables of the system, such as the total energy, free energy, entropy, and 
pressure, can be expressed in terms of the partition function or its derivatives0 

The partition function of a canonical ensembl^ is defined as a sum Z{(3) = 
J2j e-^^^ where /3 = l/iksT) is the ' inverse temperature', where T is an ordinary tem- 
perature and kB is the Boltzmann's constant. However, as the position Xj and momentum 
Pi variables of an ilh. particle in a system can vary continuously, the set of microstates is ac- 
tually uncountable. In this case, some form of coarse-graining procedure must be carried 
out, which essentially amounts to treating two mechanical states as the same microstate if 
the differences in their position and momentum variables are 'small enough'. The partition 
function then takes the form of an integral. For instance, the partition function of a gas 
consisting of N molecules is proportional to the 6 A^— dimensional phase-space integral. 



where H = H{pi,Xi), {i = 1,..., A^) is the classical Hamiltonian (total energy) function. 

''There are actually several different types of partition functions, each corresponding to different types 
of statistical ensemble (or, equivalently, different types of free energy.) The canonical partition function 
applies to a canonical ensemble, in which the system is allowed to exchange heat with the environment at 
fixed temperature, volume, and number of particles. The grand canonical partition function applies to a 
grand canonical ensemble, in which the system can exchange both heat and particles with the environment, 
at fixed temperature, volume, and chemical potential. Other types of partition functions can be defined 
for different circumstances. 

''A canonical ensemble is a statistical ensemble representing a probability distribution of microscopic 
states of the system. Its probability distribution is characterized by the proportion pi of members of the 
ensemble which exhibit a measurable macroscopic state i, where the proportion of microscopic states for 
each macroscopic state i is given by the Boltzmann distribution. 



where Ei is the energy of state i. It can be shown that this is the distribution which is most likely, if 
each system in the ensemble can exchange energy with a heat bath, or alternatively with a large number 
of similar systems. In other words, it is the distribution which has maximum entropy for a given average 
energy < Ei >. 




i„--Bi/(fcr) _ „~(E,-A)/(fcT) 
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More generally, the so-called configuration integral, as used in probability theory, in- 
formation science and dynamical systems, is an abstraction of the above definition of a 
partition function in statistical mechanics. It is a special case of a normalizing constant in 
probability theory, for the Boltzmann distribution. The partition function occurs in many 
problems of probability theory because, in situations where there is a natural symmetry, 
its associated probability measure, the Gibbs measure (see below), which generalizes the 
notion of the canonical ensemble, has the Markov property. 

Given a set of random variables Xi taking on values Xi , and purely potential Hamilto- 
nian function H{xi), {i = 1, A^), the partition function is defined as 



Z(/3) = ^exp [-PH{xi 



The function H is understood to be a real- valued function on the space of states {Xi , X2, • • • } 
while /? is a real- valued free parameter (conventionally, the inverse temperature). The sum 
over the Xi is understood to be a sum over all possible values that the random variable 
Xi may take. Thus, the sum is to be replaced by an integral when the Xi are continuous, 
rather than discrete. Thus, one writes 



Z{I3) = J dxiexp[-l3H{xi)], 



for the case of continuously- varying random variables Xi. 

The Gibbs measure of a random variable Xi having the value Xi is defined as the 
probability density function 

PiX. = X.) = ^ exp [-pEixO] = 



Zm " ' E.,exp[-/?//(x,)]- 

where E{xi) = H{xi) is the energy of the configuration .Tj. This probability, which is 
now properly normalized so that < P{xi) < 1, can be interpreted as a likelihood that a 
specific configuration of values Xi, {i = 1,2, ...N) occurs in the system. 

As such, the partition function Z{^) can be understood to provide the Gibbs measure 
on the space of states, which is the unique statistical distribution that maximizes the 
entropy for a fixed expectation value of the energy, 

dlogjZiP)) 
= dp ■ 

The associated entropy is given by 

5 = - ^ P(xi ) In P(xi ) = ) + log Z{p) . 
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The principle of maximum entropy related to the expectation value of the energy {H), 
is a postulate about a universal feature of any probability assignment on a given set of 
propositions (events, hypotheses, indices, etc.). Let some testable information about a 
probability distribution function be given. Consider the set of all trial probability distri- 
butions which encode this information. Then the probability distribution which maximizes 
the information entropy is the true probability distribution, with respect to the testable 
information prescribed. 

Now, the number of variables Xi need not be countable, in which case the set of 
coordinates {xj} becomes a field (/> = (/>(x), so the sum is to be replaced by the Euclidean 
path integral (that is a Wick-rotated Feynman transition amplitude in imaginary time), 
as 

Z{(t>) = I VicP] exp [-Hie/))] . 

More generally, in quantum field theory, instead of the field Hamiltonian H{(/)) we have 
the action 5'((/>) of the theory. Both Euclidean path integral, 

Z{(p) = J exp [— 5'((/))] , real path integral in imaginary time (11) 

and Lorentzian one, 

Z{(j)) = J I'[<?5<] exp [iS'(0)] , complex path integral in real time, (12) 



are usually called 'partition functions'. While the Lorentzian path integral (jl2p repre- 
sents a quantum-field theory-generalization of the Schrodinger equation, the Euclidean 
path integral (llip represents a statistical-field-theory generalization of the Fokker-Planck 
equation. 
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